Comments about the article in Nature: Cooperative AI: machines must learn to find common ground

Following is a discussion about this article in Nature Vol 593 6 May 2021 page 33, by Allan Dafoe e.a.
To study the full text select this link: https://www.nature.com/articles/d41586-021-01170-0 In the last paragraph I explain my own opinion.

Contents

Reflection


Introduction

Artificial-intelligence assistants and recommendation algorithms interact with billions of people every day, influencing lives in myriad ways, yet they still have little understanding of humans.
Billions of people are influenced every day, for example by reading a book or a newspaper, but that does not mean that books or newspapers understand humans. The same with any computer program. Humans even often don't understand each other.
Self-driving vehicles controlled by artificial intelligence (AI) are gaining mastery of their interactions with the natural world, but they are still novices when it comes to coordinating with other cars and pedestrians or collaborating with their human operators.
"are gaining mastery of their interactions with the natural world" I doubt that.
The state of AI applications reflects that of the research field.
Okay.
As is evident from introductory textbooks, the canonical AI problem is that of a solitary machine confronting a non-social environment.
This sentence is difficult to understand.
The basic problem is that any computer program is not intelligent or as intelligent as its designer and executes the tasks defined in each step of the program.
An AI agent — much like an infant — must first master a basic understanding of its environment and how to interact with it.
The question is how does an infant do that? That is already someting difficult to understand by humans and even more difficult to perform by a program, implemented in a baby-robot.
Even in work involving multiple AI agents, the field has not yet tackled the hard problems of cooperation.
How do people cooperate?
Most headline results have come from two-player zero-sum games, such as backgammon, chess, Go and poker.
Games like chess in some sense are very simple, in the sense that they follow very strict mathematical rules. The real question to ask is: How do they learn. . The true question to ask is how can a computer program become more intelligent without changing the program.

AI needs social understanding and cooperative intelligence to integrate well into society.
That is easy to write but difficult to solve in a mathematical way. First you must trully understand what the sentence means in all its details, and that is already difficult.

We need to build a science of cooperative AI.
That is easy to write but difficult to solve in a mathematical way.
Just as psychologists studying humans have found that the infant brain does not develop fully without social interaction, progress towards socially valuable AI will be stunted unless we put the problem of cooperation at the centre of our research.
We all know that cooperation between humans is very important to reach a certain goal, compared with the same task to be performed by a single persoon. One of the most important differences are the skills of the individuals part of the team compared with the required skills of a single persoon.

1. From autonomy to cooperation

Parents encourage their children to grow beyond their dependencies and become autonomous.
It is both: to be dependent and to be autonome
Most of the value from self-driving vehicles will come not from driving on empty roads, but from vehicles coordinating smoothly with the flow of pedestrians, cyclists and cars driven by humans.
This seems logical. The question is: Is this possible and who is responsible if there are collisions.
Thus, cooperative intelligence is not an alternative to autonomous intelligence, but goes beyond it.
Cooperation of humans, working together, to reach a common goal is much more complex than individuals reaching each, their own goals.
To replace all humans by robots, which each robot performing an autonomous task, is also rather simple.
To replace some humans by a robot is a more complex situation. Such a solution requires communication between all partners (robots and humans) involved. The complexity depents about the freedom of the robots.
AI research on cooperation will need to bring together many clusters of work.
1. A first cluster consists of AI–AI cooperation,
2. A second is AI–human cooperation, for which we will need to advance natural-language understanding
3. A third cluster is work on tools for improving human–human cooperation
The first cluster is the most difficult because it involves robot to robot communication from the outside, but because each robot actual performs the tasks embedded in a program, from the inside it involves program to program communication. The issue is: How intelligent is each program.

2. AI-AI cooperation

Four elements of cooperative intelligence

In most settings, people’s incentives are not fully aligned. The cooperative intelligence needed to achieve this has four parts:
1. Understanding.
The ability to take into account the consequences of actions, to predict another’s behaviour, and the implications of another’s beliefs and preferences.
This sentence describes the definition of the word Understanding . A detailed understanding of the sentence is already difficult among humans. For example what are our beliefs and preferences. To use this definition as part of an AI implementation is almost impossible and not practical. Two people A and B understand each other, in the sense that they can have a discussion:
  • if A can explain his opinion, B can repeat A's opinion and A agrees with what B says is his opinion.
  • if B can explain his opinion, A can repeat B's opinion and B agrees with what A says is his opinion.
Now the discussion can start in a sequence of steps:
  • A explains what the difference is between his opinion and B's opinion.
  • B explains what the difference is between his opinion and A's opinion.
  • Both A and B have to agree that this is correct.
  • A gives more information about his opinion and why he disagrees with B's opinion.
  • B gives more information about his opinion and why he disagrees with A's opinion.
  • A gives his new opinion
  • B gives his new opinion
This whole sequence repeats itself untill both A's and B's opinion are the same.

What this demonstrates is a discussion between 2 people, however it can also be a discussion between one human and one computer, but also between two computers.

2. Communication.
The ability to explicitly and credibly share information with others relevant to understanding behaviour, intentions and preferences.
This is already difficult between humans, but much more difficult when AI (i.e. robots and comptuter programs) are involved.
3. Commitment.
The ability to make credible promises when needed for cooperation.
The same
4. Norms and institutions.
Social infrastructure — such as shared beliefs or rules — that reinforces understanding, communication and commitment.
The same
Team games such as robot soccer need players on a team to work as one, jointly planning their moves and passing the ball.
I expect that all the robots in one team have the same software. I also expect that all the software is very complex and typically soccer related. (where are my team mates, where are my opponents, What do I do when I have the ball, What do I do when I don't have the ball) At the same time all robots should be different to perform a different task, that means each robot has it own set of parameters for example is initial position.
The most important question to answer is: What happens if a team looses.
Research avenues include building AIs that can understand what teammates are thinking and planning; communicate plans; and even cooperate with different kinds of teammate who might think differently and react more slowly
One thing we should not forget: all the soccer players are robots and cannot think or plan like humans do. I expect that each team has a certain set of strategies, each having a number. The problem of course is that this can lead to collisions between the players which should be prevented. At the same time each player should also have its own strategy. All in all this is a very complex problem to solve.

Yet because these situations are restricted to a perfect harmony of interests, they represent the easy case for cooperation.
I doubt that. In soccer there is only one player who can make a goal. All the other players should help him. At the same time the player who can make a goal can change depending what is happening. This makes any strategy complex.
Real-world relationships almost always involve a mixture of common and conflicting interests.
The problem with Real-world relationships is that they are never properly described. Often in politics there are many competing interests between all the different parties involved.
AI agents will need to learn how to manage these harder cooperation problems, as humans do.
That is the theory.
Most humans cannot manage how to coordinate even their own department

3. Human-AI cooperation

AI is increasingly present, underlying everything from dynamic pricing strategies to loans and prison-sentencing decisions.
What does it mean that 'computers' are used in sentencing prisoners?. What is the difference and who is responsible?
Suppose my loan is rejected by a bank. Who can explain why and who is responsible?
Real-world cooperation problems often involve multiple stakeholders, some conflicting interests and integration with our institutional and normative infrastructure.
The problem with Real-world problems is that they cannot be described in simple lanquage nor expressed in any any logical or mathematical language, as such they cannot be solved by AI.
A particular challenge facing researchers working on human–AI cooperation is that it involves, well, humans.
True?
For example, the program AlphaZero learnt to play chess by playing 44 million games against itself over 9 hours.
That is a hugh performance, but, assuming the program only knows how to move each piece, the intelligence of the program is close to zero.
Any way the program dit not 'learn' anything.
By contrast, humans produce data slowly, and require researchers to consider compensation, ethics and privacy.
These concepts are already difficult to use for simple communication between humans. We all have a different understanding what privacy means and involves. When computers are involved the resusults most probably are chaotic.
However, unlike systems that have tight integration with human workers, autonomous systems might pose greater safety risks.
A factory which is completely automated (and tested) can be very safe in operation.
Human-like AI might be more likely to displace labour.
It is already common practice to replace labour by machines. These machines can also be computer controlled. When these computers (process control plants) include AI there is a chance that they become less save.
As Stanford University radiologist Curtis Langlotz put it: “AI won’t replace radiologists, but radiologists who use AI will replace radiologists who don’t.”
The use of the word "AI" is tricky. A dentist can use X-ray as a diagnostic tool. A hugh database, with examples, can advice the dentist what the best strategy is. Anyway this is still an advice. To make this advice a must requires expert human input. To perform the whole operation automatic is a whole different ball-game.

4. AI for human collaboration

Building healthy online communities is challenging; just as social media can connect us, so too can it polarize, stress, misinform, distract and addict us.
Many care homes to day use robots for simple activities and for communication. When you ask the inhabitants they like these robots. The important question to ask is, if that what the inhabitants of these care homes really want: humans or robots. If they prefer humans than it should be made obligatoire that every human (around the age of 20) at least spend one year in a care home, as a forgoingpayment to a benefit that they themself will receive later in their life.
Researchers and developers need to find better ways to name and measure desirable properties and build algorithms that encourage them.
The question is if these algorithms are necessary. Maybe a more human oriented approach is more desirable.

5. Next steps

To succeed, cooperative AI must connect with the broader science of cooperation, which spans the social, behavioural and natural sciences.
To succeed with what?
If you are convinced that cooperation is important than first you have to understand how cooperation influences the social, behavioural and natural sciences in relation with humans only.
The need for interdisciplinarity is exemplified by a landmark work: Robert Axelrod’s The Evolution of Cooperation, published in 1984
To cooperate, to work as a team or not as a team is an interesting field of study.
Axelrod, a political scientist, brought together game theorists, mathematicians, economists, biologists and psychologists in a tournament to help devise the best algorithms for the iterated Prisoner’s Dilemma, the canonical example of why two rational people might not cooperate.
Generaly speaking the Cooperation problem cannot be solved by any form of of mathematics. Cooperation is not a game which evolves following stict rules. Only when the rules are described as something pure logical or mathematical it can be solved, implying the best strategy to win, by using methematics.
In the original question, two rational people might not cooperate. Rational implies that they use some sort of reasoning. What happens in case two criminals are used?
The Prisoner's Dilemma is a game, which follows some strict rules. The usefulness in this discussion about cooperation, is very limited.
Axelrod’s tournament offered another lesson. It gave researchers a benchmark for success in the design of cooperative algorithms, just as ImageNet did for computer vision by collecting and labelling millions of photos.
Remember the prisonners dilemma is a game and that is something completely different than clasifying photo's accordingly to certain rules.
The most important challenges of cooperation might be the most difficult to benchmark; they involve creatively stepping out of our habitual roles to change the ‘game’ itself.
This depents what you want.
Indeed, if we are to take the social nature of intelligence seriously, we need to move from individual objectives to the shared, poorly defined ways humans solve social problems: creating language, norms and institutions.
Yes. But if you can not clearly define these issues, than any form of automation will not be helpfull.
This smells like: carbage in is carbage out.
The crucial crises confronting humanity are challenges of cooperation: the need for collective action on climate change, on political polarization, on misinformation, on global public health or on other common goods, such as water, soil and clean air.
I agree.
As the potential of AI continues to scale up, a nudge in the direction of cooperative AI today could enable us to achieve much-needed global cooperation in the future.
We need sometype of worldwide cooperation between human activities. but I doubt if any form of AI will be helpfull in deciding what to do and where.


Reflection 1 - What are the limits of Articial Intelligence or AI

AI intelligence is like every computer program a sequence of instructions or steps which when executed perform a sequence of calculation as embedded in the program (instructions) and as defined by the persons who wrote (designed) the program. In short the program executes a sequence of mathematical calculations.
The general rule is when a program is executed again, using the same inputs, the results will be the same. This is the case when the program does not depend on the results on a random number generator nor is time critical.
This is also the case if you play two games of chess against a computer and in the second match you repeat exactly the same moves as in the first game. The result will be the same. What that means is that the computer has nothing learned from the first game.
If your moves are different in both games than 'ofcourse' the computer is forced to make also different moves.

A very interesting game to study AI is the game called Paint Monsters .
For an introduction about the game "Paint Monster" and about the complexity, select this link: A critical evaluation of the program Paint Monsters versus AI.. This previous written document also discusses: human intelligence versus artificial intelligence.
The name suggests that it is simple and is played by children. The reality is, that it is not. The game consists of 1605 individual games. When you have solved game #1, you can start with game #2 etc. Each game consists of moves in which the player makes certain selections. There exists a maximum number of moves. When the maximum number of moves is reached the game is terminated. The game is also terminated when a bomb explode etc. The next step for the player is to repeat the game, as many times as she likes, untill she has finally solved the game.
What makes this game interesting, is because it challenges the concept of human intelligence.
My definition of itelligence is: the individual capability of the human brain to remember what has 'happened' in the past and as a result in the future, using the logic inherent in my brain, performing the same tasks differently. In short: my individual intelligence, my logical capability, has improved. This is permanent.
When you compare this definition with a computer, or with the brain of a robot, than a computer consists of two parts: A logical section and a memory.

In order to chalenge who is the most intelligent we are going to play the game "Paint Monsters" with a human and with a computer program. That means computer program simulates a human player. In that case there is no human player involved but an opertor. The operator performs the moves as indicated by the computer program.
From a logical point of view you could say that a robot will play the game "Paint Monsters". The interface between the robot and the game is than an operator, which takes care for the communication between the two.

However to play an honest game neither the human player nor the engineer, who designed the computer program must have any knowledge in how to play the game "Paint Monsters"
When a human plays each game he will slowly learn, partly from his errors. As such his skill will improve. What the player should also do is to write down in how many tries he finaly solved each game. This allows for an honnest evaluation of the two players. To write down the total number of moves is an option.
What the engineer can do is to store all what he wants in the memory of the computer and use that data to make decissions on how to play the next game.
What the engineer can not do is to make modifications to the program, at least not make modifications to that part which make the decisions how to perform or to calculate the next move. What the enigineer also cannot do is to ask for advice over the World Wide Web. The human player is not allowed to do the same.
What the human player will experience at certain instances, when he performs a new game that also new monsters or targets (like bomb or arrow) are introduced. Such a new target can be a butterfly. Each time when a new monster is introduced normally there is a training game to explain the functionality of that new monster. What is important for the engineer that all that imformation will be available, so that in effect he can implement all the monster, in advance, during the design and initially testing of his simulation program. Those training games are not part of the evolution of either player.

My own prediction is that it is impossible to write one general program, when the designer of the computer program has on limited knowledge about what lies ahead of him, to finish all 1605 games, as implemented by the designers of the game "Paint Monsters". A more realistic goal is to write 1605 special purpose programs to finish each game. That is still not easy, because to define the specific strategies involved in each game is already tricky.

To write a computer program to play chess and beat a human player is much simpler if you have unlimited resources to calculate the next move. What makes this also simpler if the number of moves ahead you can calculate is also unlimited, including the number of processors. The point is that all these issues have nothing to do with the strategy to calculate the next move, which stays the same. In fact a computer which has more processors is not more intelligent.

Going back to the Paint Monster game it is much more challeging by two engineers to write each a simulation program and to try to find out which simulation is the most intelligent.


Reflection 2 - Paint Monsters game 1145

  1. The game starts with 5 lives. The game should be finished in 33 moves. The objective is to collect 5 arrows and 10 bombs. The field size is 9 by 7.
  2. When you start the game you get an hint: When you connect 5 monsters with the same color your reward is an arrow
  3. In move 1 5 purple monsters are connected. This creates purple arrow #1.
  4. In move 2 3 purple monsters are connected. This creates 1 spider which has 3 lives.
    The spider has to be removed within the next 3 moves, else the spider will explode and creates a web.
  5. In move 3 5 red monsters. The spider is removed and creates red arrow #2
  6. In move 4 5 blue monsters are connected. This creates blue detonator #1
  7. In move 5 4 red monsters and red arrow #2 are connected. This creates red detonator #2
    Red arrow #2 is removed. The objective is now 4 arrows and 10 bombs.
  8. In move 6 5 green monsters are connected. This creates green detonator #3
  9. In move 7 5 blue monsters are connected. This creates blue arrow #3. This also creates 1 spider which has 3 lives.
  10. In move 8 3 blue monsters and blue arrow #3 are connected. The spider is removed
    Blue Arrow #3 is removed. The objective is now 3 arrows and 10 bombs.
  11. In move 9 2 blue monsters and blue detonator #1 are connected. As a result 3 bombs explode.
    The objective is now 2 arrows and 7 bombs.
  12. After 27 moves The objective is 0 arrows and 2 bombs
  13. In move 27 2 green monsters and 1 detonator are connected. As a result 2 bombs explode
    The objective is now 0 arrows and 0 bombs and the game is finished. I still have 5 lives. The score is 9820 points.
After each move the empty fields are filled with the fields above. From a vissible point of view the fields move downwards. The empty at the top are filled with newly created monsters.


Reflection 3 - Artificial intelligence and understanding Physics

The question to ask is: if and to what extend AI can help us to understand or explain physical (chemical, biological) processes.
My understanding is, that all our understanding about the evolution of all natural processes comes from making observations and performing experiments. These observations and measurements are performed by using tools and these tools can include any type of measurement device like flow meters, pressure meters or a gas chromatography spectrometer or devices which use AI like deep learning. In all these cases the tools used should be tested and calibrated in order to demonstrate that what is indicated is accurate. From a principle point of view each tool can be handled as a black box, that means its internal functioning is 'not important'.

The question to answer is can AI be used to understand the details of a physical program.
When you consider a chemical process in its details, it is often a collection of chemical reactions which each take place depending about local temperatures, pressures or concentrations about the chemical elements involved. No AI can be used to describe these individual reactions.
In certain chemical reactions two photons are created. Photons are interesting because they can be polarised, which is consequence of the wave function of each photon. What makes certain chemical reactions specific of interest, is that when both photons are measured in the x direction, one photon will be +x and the other photon will be -x. That means the polarisation angle are each other opposite, the two photons are correlated. This is also called entanglement.
How can this correlation be explained. The question is can we use AI to explain this. I doubt that.
Suppose there are two explanations.

Which one is correct? IMO the second one because from a physical point of view, this is the simplest explanation.
Can AI help us to solve this physical problem? What you can do is to use all available litterature and count how many times solution 1 is indicated and how many times solution 2. Gives that the right answer?
IMO it is better to limit yourself to the documents who mention both solutions.

The central question of this reflection is: Can AI be used to improve our understanding of physical processes? To improve our understanding is primarily based on new experiments. To interpret these results is a human activity. To propose what to do next (and to make possible suggestions) is a human activity.
This is different than to compare the result of one experiment with the results of other experiments. Such a comparison can be automated.

If you want to give a comment you can use the following form Comment form


Created: 1 June 2021

Back to my home page Index
Back to Nature comments Nature Index